Melioration of color calibration, goal detection and self-localization systems of NAO humanoid robots

Selle lõputöö teemaks on autonoomsete robotite jalgpalli tarkvara arendamine.Vaatluse all on teemad nagu värvide kalibreerimine, objetkituvastus ja lokaliseerimine. Uus YUV värviruumi põhine automaatne värvide kalibreerimine on pakutud. Esitatakse detailne kirjeldus automaatse värvide kalibreerimise algoritmi implemenmtreerimisest koos visuaalsete näidetega, mis illustreerivad algoritmi toimimist. Samuti räägitakse täpsemalt muutustest, mis on implementeeritud väravate tuvastamise moodulis ja põhjustest nende muudatuste taga, andes hea ülevaate objekti tuvastamise algoritmi loogikast. Kirjeldatakse hetkel kasutatavat lokaliseerimissüsteemi ja pakutakse välja ning seletatakse lokaliseerimissüsteem parandamise tehnikat.In this thesis, work regarding to autonomous robot soccer software development is presented. The work covers color calibration, object detection and robot localization topics. Novel YUV color space based method for the automation of color calibration is proposed. Detailed description of automatic color calibration technique implementation is provided along with the visual results illustrating performance of the method. Changes implemented to the goal detection module and motivation behind them are described in detail, providing good overview of the logic of the object recognition algorithm. Utilised localisation system is also described and, finally, the localization system enhancement technique is proposed and explained

Articulated Object Tracking from Visual Sensory Data for Robotic Manipulation

Roboti juhtimine liigestatud objekti manipuleerimisel vajab robustset ja täpsetobjekti oleku hindamist. Oleku hindamise tulemust kasutatakse tagasisidena vastavate roboti liigutuste arvutamisel soovitud manipulatsiooni tulemuse saavutamiseks. Selles töös uuritakse robootilise manipuleerimise visuaalse tagasiside teostamist. Tehisnägemisele põhinevat servode liigutamist juhitakse ruutplaneerimise raamistikus võimaldamaks humanoidsel robotil läbi viia objekti manipulatsiooni. Esitletakse tehisnägemisel põhinevat liigestatud objekti oleku hindamise meetodit. Me näitame väljapakutud meetodi efektiivsust mitmel erineval eksperimendil HRP-4 humanoidse robotiga. Teeme ka ettepaneku ühendada masinõppe ja serva tuvastamise tehnikad liigestatud objekti manipuleerimise markeerimata visuaalse tagasiside teostamiseks reaalajas.In order for a robot to manipulate an articulated object, it needs to know itsstate (i.e. its pose); that is to say: where and in which configuration it is. Theresult of the object’s state estimation is to be provided as a feedback to the control to compute appropriate robot motion and achieve the desired manipulation outcome. This is the main topic of this thesis, where articulated object state estimation is solved using visual feedback. Vision based servoing is implemented in a Quadratic Programming task space control framework to enable humanoid robot to perform articulated objects manipulation. We thoroughly developed our methodology for vision based articulated object state estimation on these bases.We demonstrate its efficiency by assessing it on several real experiments involving the HRP-4 humanoid robot. We also propose to combine machine learning and edge extraction techniques to achieve markerless, realtime and robust visual feedback for articulated object manipulation

Real-Time Automatic Colour Calibration for NAO Humanoids

A challenge in NAO soccer robots is colour calibration. Good colour calibration can result in robust and accurate self-localization of the robot. Currently manual calibration is the only solution, which is used. In this paper, we are proposing an automatic real-time, accurate YUV colour space based colour calibration technique. In order to define average values for the desired colour classes namely orange, white, green and purple, a specified set of frames from the NAO camera are analysed. These average values are corrected by luminance analysis of a new frame and are passed to the K-means clustering algorithm as a set of initial means. In addition to these four values, a set of initial means of the K-means algorithm contains 16 values that are calculated in the following manner: the frame being processed is divided into 4 by 4 grids and the average value from every grid serves as an initial mean for K-means clustering. Consequently, colours of a similar type are combined into clusters. The final step of the proposed technique is cluster classification in which the average values of the desired colour classes are corrected by luminance analysis. As NAO cameras provide video streams in YUV format and the proposed algorithm uses this format there is no need for additional computational steps for conversation between the colour spaces. As a result, computational process is reduced compared to current techniques

Multi-contact Planning on Humans for Physical Assistance by Humanoid

International audienceFor robots to interact with humans in close proximity safely and efficiently, a specialized method to compute whole-body robot posture and plan contact locations is required. In our work, a humanoid robot is used as a caregiver that is performing a physical assistance task. We propose a method for formulating and initializing a non-linear optimization posture generation problem from an intuitive description of the assistance task and the result of a human point cloud processing. The proposed method allows to plan whole-body posture and contact locations on a task-specific surface of a human body, under robot equilibrium, friction cone, torque/joint limits, collision avoidance, and assistance task inherent constraints. The proposed framework can uniformly handle any arbitrary surface generated from point clouds, for autonomously planing the contact locations and interaction forces on potentially moving, movable, and deformable surfaces, which occur in direct physical human-robot interaction. We conclude the paper with examples of posture generation for physical human-robot interaction scenarios

Medical robots with potential applications in participatory and opportunistic remote sensing: A review

Among numerous applications of medical robotics, this paper concentrates on the design, optimal use and maintenance of the related technologies in the context of healthcare, rehabilitation and assistive robotics, and provides a comprehensive review of the latest advancements in the foregoing field of science and technology, while extensively dealing with the possible applications of participatory and opportunistic mobile sensing in the aforementioned domains. The main motivation for the latter choice is the variety of such applications in the settings having partial contributions to functionalities such as artery, radiosurgery, neurosurgery and vascular intervention. From a broad perspective, the aforementioned applications can be realized via various strategies and devices benefiting from detachable drives, intelligent robots, human-centric sensing and computing, miniature and micro-robots. Throughout the paper tens of subjects, including sensor-fusion, kinematic, dynamic and 3D tissue models are discussed based on the existing literature on the state-of-the-art technologies. In addition, from a managerial perspective, topics such as safety monitoring, security, privacy and evolutionary optimization of the operational efficiency are reviewed

Maximizing Performance with Minimal Resources for Real-Time Transition Detection

Assistive devices, such as exoskeletons and prostheses, have revolutionized the field of rehabilitation and mobility assistance. Efficiently detecting transitions between different activities, such as walking, stair ascending and descending, and sitting, is crucial for ensuring adaptive control and enhancing user experience. We here present an approach for real-time transition detection, aimed at optimizing the processing-time performance. By establishing activity-specific threshold values through trained machine learning models, we effectively distinguish motion patterns and we identify transition moments between locomotion modes. This threshold-based method improves real-time embedded processing time performance by up to 11 times compared to machine learning approaches. The efficacy of the developed finite-state machine is validated using data collected from three different measurement systems. Moreover, experiments with healthy participants were conducted on an active pelvis orthosis to validate the robustness and reliability of our approach. The proposed algorithm achieved high accuracy in detecting transitions between activities. These promising results show the robustness and reliability of the method, reinforcing its potential for integration into practical applications.Comment: Submitted for a conference. 7 pages including references, 8 figures, 3 table

Block based image compression technique using rank reduction and wavelet difference reduction

In this paper a new block based lossy image compression technique which is using rank reduction of the image and wavelet difference reduction (WDR) technique, is proposed. Rank reduction is obtained by applying singular value decomposition (SVD). The input image is divided into blocks of equal sizes after which quantization by SVD is carried out on each block followed by WDR technique. Reconstruction is carried out by decompressing each blocks bit streams and then merging all of them to obtain the decompressed image. The visual and quantitative experimental results of the proposed image compression technique are shown and also compared with those of the WDR technique and JPEG2000. From the results of the comparison, the proposed image compression technique outperforms the WDR and JPEG2000 techniques

Aide à domicile au moyen de robots humanoïdes : assistance au mouvements en interaction physique multi-contacts

As the percentage of the elderly population is rising worldwide, the demand imposed on modern society for providing a sufficient amount of skilled workers in the caregiving sector is becoming increasingly harder to fulfill. Utilization of robotic technologies in physical assistance in a home support context can contribute to sustaining a frail person's autonomy and quality of life. We envision the use of the humanoid robot technology for providing daily assistance with physical motion tasks. More specifically, in this work, we focus on the usage of Pepper humanoid robot platform, mass-produced by SoftBank Robotics. The choice of the platform is motivated by its affordability, user-friendly design and multi-modal communication capabilities.First, we propose and develop a proprioceptive sensor based contact detection method. In order to maintain low cost of the platform, our method for contact detection aims at using only the available Pepper’s sensors to detect collision with the environment, namely a contact event during physical human-robot interaction. We detail the integration of the proposed method as a feedback signal in the whole-body controller to react to human touch in real-time.Secondly, we investigate the whole-body humanoid robot posture planning problem in the assistive physical human-robot interaction context. We augment the non-linear optimization based posture generation framework with necessary components that allow us to plan a robot attitude in contact with a human point cloud. The proposed human point cloud processing pipeline provides the necessary data structures to formulate the posture generation problem for a robot to initiate a physical assistance task.Then, we present a fully autonomous interaction scenario for initiating a physical assistance process. A Finite-State Machine and task-space Quadratic Programming based controller is developed for a robot to navigate towards a human, perform a multi-modal communication and establish several physical contacts in a fully autonomous fashion. The controller performance is demonstrated in real experiments with a human subject. All the software tools developed to perform whole-body task space Quadratic Programming based control on SoftBank humanoid robots are made publicly available and are documented in detail.Finally, we study the problem of partial physical assistance in motion. We present a control methodology that enables a humanoid robot to supply the assistive forces necessary to help a frail human to achieve a desired performance of a motion task. We present and discuss the simulation results of the proposed method.We conclude this work with discussion of the achieved results and the future perspectives of research in the area of humanoid-human interaction for physical assistance.Alors que le pourcentage de seniors parmi la population mondiale augmente, la quantité de personnel soignant qualifié pour l’aide à cette catégorie de personnes est elle en perpétuelle diminution. Cette thèse défend l’idée que l'utilisation des technologies robotiques pour une assistance physique pourrait contribuer à maintenir l'autonomie et la qualité de vie des personnes fragiles, et par conséquent un maintien à domicile plus long. Les robots humanoïdes peuvent prendre part à une telle vision, notamment pour effectuer les tâches à valeurs non ajoutées pour le personnel soignant ou la famille. Au cours de ce travail, une attention plus particulière est portée à l’utilisation de la plateforme humanoïde, Pepper, premier robot humanoïde produit en grande série. Le choix de cette plateforme est motivé par son accessibilité lié à son coût mais également par sa conception voulue sociale, qui le rend convivial ; ses capacités de communication multimodales facilitent grandement certaines tâches. Dans ce cadre, nous avons développé une méthode de détection de contact basée sur les capteurs proprioceptifs. Afin de maintenir le coût du robot, la détection du contact utilise uniquement les capteurs déjà présents sur le robot. Il s’agit pour le robot de détecter les collisions avec l’environnement, et plus spécifiquement la détection de l’interaction physique homme/robot. L’intégration de la méthode proposée passe par l’analyse du signal de retour des capteurs pour ajuster la réponse en temps réel du robot à l’événement de contact détecté.Ensuite, nous avons abordé le problème de la planification de la posture des robots humanoïdes dans le contexte de l'interaction physique homme-robot. Nous avons revu le framework de génération de posture basé sur l'optimisation non linéaire avec les composants nécessaires qui permettent de planifier une posture en contact avec un nuage de points issu de la perception de la personne à assister. Le pipeline de traitement du nuage de points proposé fournit les structures de données nécessaires pour formuler le problème de génération de posture pour qu'un robot puisse initier une tâche d'assistance physique.Suite aux discussions avec un centre EHPAD, un premier scénario d'interaction entièrement autonome est proposé pour initier le processus d'assistance. Un contrôleur dans l’espace des tâches formulé comme un programme quadratique est développé pour que Pepper puisse atteindre une personne, effectuer une communication multimodale et établir plusieurs contacts physiques de manière totalement autonome. La performance du contrôleur est démontrée par une expérience réelle. Tous les outils logiciels développés pour effectuer le contrôle du corps entier des robots humanoïdes de SoftBank sont mis à la disposition du public et sont documentés en détail.Enfin, nous avons entamé le problème de l'assistance physique à des mouvements prédéfinis. Nous présentons une méthodologie de contrôle adaptative qui permet au robot Pepper de fournir les forces d'assistance nécessaires pour accompagner un mouvement effectué (ici le bras) par une personne avec une suppléance des couples articulaires. Nous présentons les résultats préliminaires pour l’approche proposée.Nous concluons notre thèse par une discussion sur les résultats obtenus et les perspectives futures de la recherche concernant l'interaction physique homme-robot pour l'assistance physique au mouvement

YUV based automatic colour calibration for NAO robots

A challenge in real time application of NAO soccer robots is in colour calibration. Many tasks such as localisation and goal detection rely on robustness of colour calibration. In this paper a robust and accurate YUV colour space based automatic colour calibration technique is proposed. First the specific set of frames from the NAO's camera has been analysed in order to define average values for desired colour classes, namely orange, white, green and purple. Then those average values are corrected by a luminance analysis of a new frame and are passed to the K-means clustering algorithm as a set of initial means. Apart from those 4 values, set of initial means of the K-means algorithm also contains 16 values that are calculated in the following manner: the frame currently being processed is divided into 4 by 4 grid and average value from every grid will serve as an initial mean for K-means clustering. After the K-means clustering is applied to the frame so that colours of a similar type are combined into clusters. Final step of the proposed technique is the cluster classification, which is performed by measuring the distance from cluster centroids to the previously calculated average values of desired colour classes corrected by luminance analysis. The proposed colour calibration technique has been tested on white goal detection

Edge information based object classification for NAO robots

This paper presents a research regarding the development of a computationally cheap and reliable edge information based object detection and classification system for use on the NAO humanoid robots. The work covers ground detection, edge detection, edge clustering, and cluster classification, the latter task being equivalent to object recognition. In this work, a new geometric model for ground detection, a joint edge model using two edge detectors in unison for improved edge detection, and a hybrid edge clustering model have been proposed which can be implemented on NAO robots. Also, a classification model is outlined along with example classifiers and used values